During editing of an audiovisual work, it is common to expand or to contract an audiovisual clip to fit a particular time slot or to create a perceptual rate change. Such operations may be performed for a creative purpose or for a technical purpose. There are a number of terms used to describe such operations, including “motion effect,” “fit-to-fill,” “time scaling,” or, generally, a “retiming effect.” If the operation has the effect of changing the perceived playback rate from one constant rate to another constant rate, then the operation may be called a “constant” retiming effect. If the operation has the effect of providing a perceived playback rate that varies over the length of the clip, then the operation may be called a “rampable” retiming effect. The variable rate change of a rampable retiming effect commonly is defined by a function curve. The function curve typically is a speed curve that describes how the perceived playback speed varies from the original playback speed, in terms of a percentage of the original playback speed.
There are many ways in which such retiming effects may be implemented. Commonly, a retiming effect is performed by stretching one of the audio or video tracks to match timing of the other track. Video and audio also typically are retimed separately. Video is retimed in video post-production using techniques such as sample and hold, blended frames or motion-based interpolation. In audio post-production, the audio may be replaced with another soundtrack, may remain unchanged or may be retimed using techniques such as time scaling or pitch shifting. Both audio retiming techniques change the perceived playback rate of the audio, but time scaling may be used to avoid or control modifying the pitch.
Because the video and audio typically are retimed separately, the video and audio typically are retimed using different speed curves or using the same function curve but sampled by different sampling rates. Either technique makes it difficult to retain synchronization between the audio and video.